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1. Introduction 


Consider a random vector Z taking values in R", drawn from the standard 
Gaussian distribution 7 , whose density is given by 
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for each x £ R™, where | • | denotes the Euclidean norm. It is well known that 
when the dimension n is large, the distribution of Z is highly concentrated 
around the sphere of radius 1 Jii; that yfn is the appropriate radius follows by 
the trivial observation that E|Z | 2 = EZ? = n. One way to express this 
concentration property is by computing the variance of |Z| 2 , which is easy to 
do using the independence of the coordinates of Z: 


Var(|Z| 2 ) = Var 



^Var(Z 2 )=2n. 

2=1 


This work was partially supported by the project GeMeCoD ANR 2011 BS01 007 01, and 
by the U.S. National Science Foundation through the grant DMS-1409504 (CAREER). 
A significant portion of this paper is based on the Ph.D. dissertation of L. Wang G2], 
co-advised by M. Madiman and N. Read, at Yale University. 




2 


Fradelizi, Madiman and Wang 


In particular, the standard deviation of \Z\ 2 is \/2n, which is much smaller 
than the mean n of \Z\ 2 when n is large. Another way to express this con¬ 
centration property is through a deviation inequality: 

p|^- - 1 >i| < exp | - log(l + t)]| (1.1) 

for the upper tail, and a corresponding upper bound on the lower tail. These 
inequalities immediately follow from Clrernoff’s bound, since \Z\ 2 /n is just 
the empirical mean of i.i.d. random variables. 

It is natural to wonder if, like so many other facts about Gaussian 
measures, the above concentration property also has an extension to log- 
concave measures (or to some subclass of them). There are two ways one 
may think about extending the above concentration property. One is to ask 
if there is a universal constant C such that 

Var(|Aj 2 ) < Cn, 

for every random vector X that has an isotropic, log-concave distribution on 
K™. Here, we say that a distribution on R™ is isotropic if its covariance matrix 
is the identity matrix; this assumption ensures that E|X | 2 = n, and provides 
the normalization needed to make the question meaningful. This question has 
been well studied in the literature, and is known as the “thin shell conjecture” 
in convex geometry. It is closely related to other famous conjectures: it implies 
the hyperplane conjecture of Bourgain mm, is trivially implied by the 
Kannan-Lovasz-Simonovits conjecture, and also implies the Kannan-Lovasz- 
Simonovits conjecture up to logarithmic terms [12| . The best bounds known 
to date are those of Guedon and E. Milman ifS], and assert that 

Var(|AT| 2 ) < CVi 4/3 . 

The second way that one may try to extend the above concentration 
property from Gaussians to log-concave measures is to first observe that the 
quantity that concentrates, namely \Z\ 2 , is essentially the logarithm of the 
Gaussian density function. More precisely, since 

- log (j>(x) = | log(27r) + ^j-, 

the concentration of \Z\ 2 about its mean is equivalent to the concentration 
of — log 4>{Z) about its mean. Thus one can ask if, for every random vector 
X that has a log-concave density / on I", 

Var(— log f(X)) < Cn (1.2) 

for some absolute constant C. An affirmative answer to this question was 
provided by Bobkov and Madiman ,3]. The approach of [3 can be used to 
obtain bounds on C , but the bounds so obtained are quite suboptimal (around 
1000). Recently V. H. Nguyen 27] (see also [23]) and L. Wang [32] indepen¬ 
dently determined, in their respective Ph.D. theses, that the sharp constant 
C in the bound is 1. Soon after this work, simpler proofs of the sharp 
variance bound were obtained independently by us (presented in the proof of 
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Theorem [231 in this paper) and by Bolley, Gentil and Guillin [7] (see Remark 
4.2 in their paper). An advantage of our proof over the others mentioned is 
that it is very short and straightforward, and emerges as a consequence of a 
more basic log-concavity property (namely Theorem 12.91) of L p -norms of log- 
concave functions, which may be thought of as an analogue for log-concave 
functions of a classical inequality of Borell [5] for concave functions. 

If we are interested in finer control of the integrability of — logf(X), 
we may wish to consider analogues for general log-concave distributions of 
the inequality CD- Our second objective in this note is to provide such 
an analogue (in Theorem 14.II) . A weak version of such a statement was an¬ 
nounced in [4] and proved in [5], but the bounds we provide in this note 
are much stronger. Our approach has two key advantages: first, the proof 
is transparent and completely avoids the use of the sophisticated Lovasz- 
Simonovits localization lemma, which is a key ingredient of the approach in 
[3j; and second, our bounds on the moment generating function are sharp, 
and are attained for example when the distribution under consideration has 
i.i.d. exponentially distributed marginals. 

While in general exponential deviation inequalities imply variance bounds, 
the reverse is not true. Nonetheless, our approach in this note is to first prove 
the variance bound CD, and then use a general bootstrapping result (Theo¬ 
rem m to deduce the exponential deviation inequalities from it. The boot¬ 
strapping result is of independent interest; it relies on a technical condition 
that turns out to be automatically satisfied when the distribution in question 
is log-concave. 

Finally we note that many of the results in this note can be extended to 
the class of convex measures; partial work in this direction is done by Nguyen 
|28j , and results with sharp constants are obtained in the forthcoming paper 

m- 

2. Optimal varentropy bound for log-concave distributions 

Before we proceed, we need to fix some definitions and notation. 

Definition 2.1. Let a random vector X taking values in M n have probability 
density function /. The information content of X is the random variable 
h(X) = — log/(X). The entropy of X is defined as h(X) = E(/i(X)). The 
varentropy of a random vector X is defined as V{X) = Var (h(X)). 

Note that the entropy and varentropy depend not on the realization of X 
but only on its density /, whereas the information content does indeed depend 
on the realization of X. For instance, one can write h(X) = — f Rn f log / and 

V(X) = Var(log/(X)) = [ /(log/) 2 -( [ /log/) . 

J K" \J R" / 

Nonetheless, for reasons of convenience and in keeping with historical con¬ 
vention, we slightly abuse notation as above. 
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As observed in [3], the distribution of the difference h(X) — h(X) is 
invariant under any affine transformation of R n (i.e., h(TX ) — h(TX) = 
h(X) — h(X) for all invertible affine maps T : R" —► R n ); hence the varentropy 
V(X) is affine-invariant while the entropy h(X) is not. 

Another invariance for both h(X) and V(X) follows from the fact that 
they only depend on the distribution of log(/(X)), so that they are unchanged 
if / is modified in such a way that its sublevel sets keep the same volume. This 
implies (see, e.g., [2S) Theorem 1.13]) that if / is the spherically symmetric, 
decreasing rearrangement of /, and X is distributed according to the density 
/, then h(X) = h(X) and V{X) = V(X). The rearrangement-invariance 
of entropy was a key element in the development of refined entropy power 
inequalities in [331 . 

Log-concavity is a natural shape constraint for functions (in particular, 
probability density functions) that represents an infinite-dimensional gener¬ 
alization of the class of Gaussian distributions. 


Definition 2.2. A function / : R” — > [0, oo) is log-concave if / can be written 
as 

f{x)=e- u ^\ 

where U : R™ (—oo,+oo] is a convex function, i.e., U{tx + (1 — t)y) < 
tU(x) + (1 — t)U(y ) for any x, y and 0 < t < 1. When / is a probability 
density function and is log-concave, we say that / is a log-concave density. 

We can now state the optimal form of the inequality OD, first obtained 
by Nguyen m and Wang [32] as discussed in Section |T| 

Theorem 2.3. [271132] Given a random vector X in R" with log-concave den¬ 
sity f, 

V(X) < n 

Remark 2.4. The probability bound does not depend on f - it is universal over 
the class of log-concave densities. 


Remark 2.5. The bound is sharp. Indeed, let X have density f = e~ v , with 
ip : R ra — > [0, oo] being positively homogeneous of degree 1, i.e., such that 
ip(tx ) = tip(x) for all t > 0 and all x G R™. Then one can check that the 
random variable Y = g>(X) has a gamma distribution with shape parameter 
n and scale parameter 1, i.e., it is distributed according to the density given 
by 


fr(t) = 


t n - 1 e~ t 
(■n — 1 )! 


Consequently E(T) = n and E(T 2 ) = n(n + 1), and therefore V(X) = 
Var(T) = n. Particular examples of equality include: 

1. The case where p{x) = Y^i =i x i on cone of points with non-negative 
coordinates (which corresponds to X having i.i.d. coordinates with the 
standard exponential distribution), and 
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2. The case where <p(x) = inf{r > 0 : x £ rK} for some compact convex set 
K containing the origin (which, by taking K to be a symmetric convex 
body, includes all norms on 1” suitably normalized so that e~ v is a 
density). 

Remark 2.6. Bolley, Gentil and Guillin [7] in fact prove a stronger inequality, 
namely, 


-l 


_I_ - I > EjVtfpO • Hess(U{X))- l VU(X)} 


This gives a strict improvement of Theorem, \2.S\ when the density f = e~ u 
of X is strictly log-concave, in the sense that Hess{U{X)) is, almost surely, 
strictly positive definite. As noted by [7], one may give another alternative 
proof of Theorem \2.S\ by applying a result of Harge in Theorem 2]. 

In order to present our proof of Theorem 12.31 we will need some lem¬ 
mata. The first one is a straightforward computation that is a special case of 
a well known fact about exponential families in statistics, but we write out a 
proof for completeness. 

Lemma 2.7. Let f be any probability density function on M" such that f £ 
L a (R n ) for each a > 0, and define 



Let X a be a random variable with density f a on M", where 



Then F is infinitely differentiable on (0,oo), and moreover, for any a > 0, 


F"(a) = \v(X a ). 

or 


Proof. Note that the assumption that / £ L“(K ra ) (or equivalently that 
F(a) < oo) for all a > 0 guarantees that F(a) is infinitely differentiable 
for a > 0 and that we can freely change the order of taking expectations and 
differentiation. 

Now observe that 



if we wish, we may also massage this to write 


F'{a) = ^[F(a) - h(X a )\. 


(2.1) 
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Differentiating again, we get 



/r(iog /) 2 f jf a iogf \ 


2 


//“ V If a ) 

J /a (log /) 2 - ^j /a log/) 


2 


= Var[log/(X a )] = Var -{log/ a (X Q ) + F(a)} 

a 


= —2 Var[log f a (X a )] = AAA 
or cr 


as desired. 


□ 


The following lemma is a standard fact about the so-called perspective 
function in convex analysis. The use of this terminology is due to Hiriart- 
Urruty and Lemarechal [SOI p. 160] (see m for additional discussion), al¬ 
though the notion has been used without a name in convex analysis for a 
long time (see, e.g., pJO] p. 35]). Perspective functions have also seen recent 
use in convex geometry mum) and empirical process theory [31]. We give 
the short proof for completeness. 

Lemma 2.8. If U : K n — > K.U {+oo} is a convex function, then 

w(z,a) :=aU(z/a) 
is a convex function on K™ x (0, +oo). 

Proof. First note that by definition, w{az,aa) = aw(z,a ) for any a > 0 and 
any (z,a) £ R" x (0,+ 00 ), which implies in particular that 



Hence 


w(Xzi + (1 — X)z 2 , Acta + (1 — A)a 2 ) 



= \w(zi, ai) + (1 - A )w{z 2 ,a 2 ), 
for any A G [0,1], Zi,z 2 £ R", and a\,a 2 £ (0, 00 ). 


□ 


The key observation is the following theorem. 
Theorem 2.9. If f is log-concave on R n , then the function 



is log-concave on (0,+ 00 ). 
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Proof. Write / = e u , with U convex. Make the change of variable a ' — z/a 
to get 

G(a) = J e- aU ^l a) dz. 

The function w(z,a ) := all (z/a) is convex on 1" x (0, +oo) by Lemma ITH1 
which means that the integrand above is log-concave. The log-concavity of 
G then follows from Prekopa’s theorem [29], which implies that marginals of 
log-concave functions are log-concave. □ 


Remark 2.10. An old theorem of Borell [5, Theorem 2] states that if f is 
concave on then Gf(p) := (p+ 1) • • • (p + n) J f p is log-concave as a func¬ 
tion of p £ (0, oo). Using this and the fact that a log-concave function is a 
limit of a-concave functions with a —> 0, one can obtain an alternate, indi¬ 
rect proof of Theorem. \2. .91 One can also similarly obtain an indirect proof of 
Theorem I£..91 by considering a limiting version of [5] Theorem VII. 2], which 
expresses a log-concavity property of (p — 1)... (p — n) J (f>~ p for any convex 
function <p on R n , for p > n+ 1 (an improvement of this to the optimal range 
p > n is described in mm, although this is not required for this alternate 
proof of Theorem roi) . 

Proof of Theorem \2.A Since / is a log-concave density, it necessarily holds 
that / £ L a (R ra ) for every a > 0; in particular, G(a ) := a" / /“ is finite and 
infinitely differentiable on the domain (0,oo). By definition, 

log G(a) = n log a + log J f a =n\oga + F(a). 

Consequently, 

^[logG(a)] =—^ + F"(a). 

By Theorem 12.91 logG(a) is concave, and hence we must have that 

-^ + F"(a) <0 
cr 

for each a > 0. However, Lemma 1^771 implies that F"(a ) = V(X a )/a 2 , so 
that we obtain the inequality 

V(X a ) - n 
a 2 

For a = 1, this implies that V(X) <n. □ 


Notice that if / = e ~ u , where U : —> [0, oo] is positively homo¬ 

geneous of degree 1, then the same change of variable as in the proof of 
Theorem 12.91 shows that 


G(a) 



f(z)dz = 1. 


Hence the function G is constant. Then the proof above shows that V (X) = n, 
which establishes the equality case stated in Remark 12.51 
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3. A general bootstrapping strategy 

The purpose of this section is to describe a strategy for obtaining exponential 
deviation inequalities when one has uniform control on variances of a family 
of random variables. Log-concavity is not an assumption made anywhere in 
this section. 


Theorem 3.1. Suppose X ~ f, where f £ L a (R n ) for each a > 0. Let 
X a ~ f a , where 


fa 0 ) 


f a (x) 

//“ 


If K = K(f) := sup Q>0 V(X a ), then 


E[ e HHX)-HX)}] < e Kr(.-P) ) 


where 


( u — log(l + u) for u >—1 
\ +oo for u < — 1 


Proof. Suppose X is a random vector drawn from a density / on R ra , and 
define, for each a > 0, F(a) = log J /“. Set 

K = supld(X Q ) = sup a 2 F"(a); 

a>0 a>0 


the second equality follows from Lermna [2.71 Since / S L Q (R”) for each a > 0, 
F(a) is finite and moreover, infinitely differentiable for a > 0, and we can 
freely change the order of integration and differentiation when differentiating 
F(a). 

From Taylor-Lagrange formula, for every a > 0, one has 

/ a 

(a — u)F" (u)du. 


Using that F( 1) = 0, F"(u) < K/u 2 for every u > 0 and the fact that for 
0 < a < u < 1, one has a — u < 0, we get 


F(a) < (a 
= (a 


l)F'(l) + K 
l)F'(l) + K 


Ji u 2 

-- - log(u) 
L u 


Thus, for a > 0, we have proved that 


F(a) < (a — 1)F 1, (1) + K(a — 1 — log a). 


Setting f3 = 1 — a, we have for /? < 1 that 

e H I-/?) < e -0F'(i) e K(-p-io g (i-p))' 

Observe that e F(1 “ /3) = / f x ~ p = E [f~ 0 (X)] = E[ e - /31 ° s/( ' Y) ] = F[e^ X) ] 
and e~P F < - 1 - ) = the latter fact follows from the fact that F'( 1) = 
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—h(X) as is clear from the identity (12.11) . Hence the inequality (13.11) may be 
rewritten as 

E [ e HHx)-h(x)^ ( 3 . 2 ) 

□ 


Remark 3.2. We note that the function r(t) = t — log(l + t) for t > — 1, 
(or the related function h(t) = tlogt — t + 1 for t > 0, which satisfies 
sh(t/s) = tri(s/t) for r\(u) = r{u — 1)) appears in many exponential con¬ 
centration inequalities in the literature, including Bennett’s inequality {Tj (see 
also ®), and empirical process theory [3U. It would be nice to have a clearer 
understanding of why these functions appear in so many related contexts even 
though the specific circumstances vary quite a bit. 

Remark 3.3. Note that the function r is convex on R and has a quadratic 
behavior in the neighborhood of 0 (r(u) ^) and a linear behavior at +oo 
(r(u) ~oc u). 

Corollary 3.4. With the assumptions and notation of Theorem \3.1i we have 
for any t > 0 that 

P{h(X) - h(X) >t} < exp | - Kr(^j J 

P {h(X) - h(X) < -t} < exp | - Kr{' - | 

The proof is classical and often called the Cramer-Chernoff method (see 
for example section 2.2 in 0)- It uses the Legendre transform ip* of a convex 
function ip : M — > R U {+oo} defined for y £ R by 

‘ P*(y ) = sup xy - <p(x). 

X 

Notice that if rniniyj = y>(0) then for every y > 0, the supremum is reached 
at a positive x, that is <p*(y) = sup x>0 xy — <p(x). Similarly, for y < 0, the 
supremum is reached at a negative x. 


Proof. The idea is simply to use Markov’s inequality in conjunction with 
Theorem 13.11 and optimize the resulting bound. 

For the lower tail, we have for /3 > 0 and t > 0, 


P [h(X) - h(X) < —t] < E 


-ft (h(X)-h(X)) 


< exp 


H* 


o-0t 


m- 


Thus minimizing on /? > 0, and using the remark before the proof, we get 
¥[h(X) — h(X) < —t] < exp | — K sup — r(/3)^ | = e~ Kr (3.3) 
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Let us compute the Legendre transform r* of r. For every t, one has 
r*(t) = sup tu — r(u) = sup (tu — u + log(l + u)) . 

U U>—1 


One deduces that r*(t) = +oo for t > 1. For t < 1, by differentiating, the 
supremum is reached at u = t/(l — t) and replacing in the definition we get 


r*(t) = — t — log(l — t) = r(—t). 


Thus r*(t) = r(—t) for all tel. Replacing, in the inequality (13.31) . we get 
the result for the lower tail. 

For the upper tail, we use the same argument: for /? > 0 and t > 0, 


¥[h(X) - h(X) >t\< E 


j(h{X)-h{X)) 


O-Pt 


p t 


< exp \ K ( r(—f$) - — 


Thus minimizing on /? > 0, we get 
P[h(X) — h(X) > t] < exp 


K sup 

p> o 




(3.4) 


Using the remark before the proof, in the right hand side term appears the 
Legendre transform of the function r defined by f(it) = r(—u). Using that 
r*(t) = r(—t) = f(t), we deduce that (f)* = (r*)* = r. Thus the inequality 
(13.41) gives the result for the upper tail. 

□ 


4. Conclusion 

The purpose of this section is to combine the results of Sections [2] and [3] to 
deduce sharp bounds for the moment generating function of the information 
content of random vectors with log-concave densities. Naturally these yield 
good bounds on the deviation probability of the information content h(X) 
from its mean h(X) = E h(X). We also take the opportunity to record some 
other easy consequences. 

Theorem 4.1. Let X be a random vector in R n with a log-concave density f. 
For P < 1, 


E 

e /3[h(X)-h(X)} 

< E 

e P[h{X*)-h(X*)] 






where X* has density f* = e 5di=i Xi , restricted to the positive quadrant. 

Proof. Taking K = n in Theorem 13.11 (which we can do in the log-concave 
setting because of Theorem 12.31) , we obtain: 

E [ e /3{h(*)-Mx)}] < e ™-(-/3) ; ^|. 
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Some easy computations will show: 

E [ e /W**)-fcpni] =e nr{- f 3) ) 

This concludes the proof. 

□ 


As for the case of equality of Theorem 12.31 discussed in Remark 12.51 
notice that there is a broader class of densities for which one has equality in 
Theorem l4.ll including all those of the form e~^ K , where K is a symmetric 
convex body. 


Remark 4.2. The assumption ft < 1 in Theorem EH is strictly not required; 
however, for ft > 1, the right side is equal to +oo. Indeed, already for ft = 1, 
one sees that for any random vector X with density f, 


E [ e h{x)-h(x) i =e -H a') e 


/P0 

= e~ h( - x) Vol n (supp(f)), 


= e~ h W 


dx 


' supp(f) 


where supp(f) = {x£ R n : /( x) > 0} is the support of the density f and Vol n 
denotes Lebesgue measure on R”. In particular, this quantity for X*, whose 
support has infinite Lebesgue measure, is +oo. 


Remark 4.3. Since 


lim — E 

a->0 or 


0 a(log/(A-)-E[log/(*)]) 


we can recover Theorem \2.3\ from Theorem EH 


V{X), 


Taking K = n in Corollary 13.41 (again because of Theorem 12.31) . we 
obtain: 


Corollary 4.4. Let X be a random vector in R" with a log-concave density f. 
For t > 0, 

P [h{X) - h{X) < -nt} < e~ nr ^~ t \ 

P \h(X) - h{X) > nt) < e~ nr< d\ 
where r{u) is defined in Th,eorem, \3.1\ 

The original concentration of information bounds obtained in [3] were 
suboptimal not just in terms of constants but also in the exponent; specifically 
it was proved there that 

P j- \h(X) - h(X)\ > 1 1 < 2e~ ct ^ (4.1) 

for a universal constant c > 1/16 (and also that a better bound with ct 2 n in 
the exponent holds on a bounded range, say, for t £ (0, 2]). One key advantage 
of the method presented in this paper, apart from its utter simplicity, is the 
correct linear dependence of the exponent on dimension. Incidentally, we 
learnt from a lecture of B. Klartag [22] that another proof of (EH) can be 
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given based on the concentration property of the eigenvalues of the Hessian 
of the Brenier map (corresponding to optimal transportation from one log- 
concave density to another) that was discovered by Klartag and Kolesnikov 
[23] : however, the latter proof shares the suboptimal y/nt exponent of [3]. 

The following inequality is an immediate corollary of Corollarv H. 41 since 
it merely expresses a bound on the support of the distribution of the infor¬ 
mation content. 

Corollary 4.5. Let X have a log-concave probability density function f on R". 
Then: 

h{X) < log H/lloo T n. 

Proof. By Corollary 14.41 almost surely, 

log f(X) < E[log f(X)]+n, 

since when t > 1, P[log/(X) — E[log/(X)] > nt] = 
over all realizable values of X yields 

log ll/lloo < E[log f{X)]+n, 

which is equivalent to the desired statement. 


0. Taking the supremum 


□ 


Corollarv H. 51 was first explicitly proved in [S], where several applications 
of it are developed, but it is also implicitly contained in earlier work (see, e.g., 
the proof of Theorem 7 in nzn- 

An immediate consequence of Corollary 14.51 unmentioned in [5], is a 
result due to (T5I : 


Corollary 4.6. Let X be a random vector in R” with a log-concave density f. 
Then 

ll/lloo < e"/(E[X]). 

Proof. By Jensen’s inequality, 

log/(EX) > E[log/(X)]. 

By Corollary 14.51 

E[log/(X)] > log ll/lloo — n. 

Hence, 

log/(EX) > log ll/lloo -n. 

Exponentiating concludes the proof. □ 


Finally we mention that the main result may also be interpreted as a 
small ball inequality for the random variable /(X). As an illustration, we 
record a sharp form of [2U Corollary 2.4] (cf., [2T] Corollary 5.1] and [B] 
Proposition 5.1]). 
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Corollary 4.7. Let f be a log-concave density on R". Then 

P{/P0 > c n ||/||oo} > 1 - (e• c• log , 

where 0 < c < j. 

Proof. Note that 

n/PO < c n ||/||oo} =P{log/(X) < log ||/1|co + n logc} 

= P (h{X) > — log H/lloo - nlogc} 

< P {h(X) > h(X) — n( 1 + logc)}. 

using Corollary 14.51 for the last inequality. Applying Corollary 14.41 with t = 
— log c — 1 yields 

P{/PO < c n ||/|| 00 } < e - nr ( _1_lo S c ). 

Elementary algebra concludes the proof. □ 

Such “effective support” results are useful in convex geometry as they 
can allow to reduce certain statements about log-concave functions or mea¬ 
sures to statements about convex sets; they thus provide an efficient route to 
proving functional or probabilistic analogues of known results in the geometry 
of convex sets. Instances where such a strategy is used include PH [B]. These 
and other applications of the concentration of information phenomenon are 
discussed in [2B’. 
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